Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix for failing nightly iOS end to end tests #6120

Merged
merged 1 commit into from
Apr 17, 2024

Conversation

niklasberglund
Copy link
Collaborator

@niklasberglund niklasberglund commented Apr 12, 2024

Fixes for testLogin and testCustomDNS end to end tests failing. This PR removes the "no time account number" which is a persistent account without any time on it, and adds teardown blocks for testAdBlockingViaDNS and testAdBlockingViaDNS reset state changes that affected other tests.

To test the changes in this PR its easiest to run the GitHub actions workflow iOS end-to-end tests. I did a run on this branch at https://github.com/mullvad/mullvadvpn-app/actions/runs/8690296877


This change is Reviewable

@niklasberglund niklasberglund added the iOS Issues related to iOS label Apr 12, 2024
@niklasberglund niklasberglund force-pushed the fix-nightly-ios-test-issues branch 3 times, most recently from 00dfcba to 1a82fa8 Compare April 15, 2024 14:06
@niklasberglund niklasberglund force-pushed the fix-nightly-ios-test-issues branch 2 times, most recently from ca84ab2 to 58f0140 Compare April 15, 2024 14:11
@niklasberglund niklasberglund marked this pull request as ready for review April 15, 2024 14:12
Copy link
Contributor

@rablador rablador left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 4 of 6 files at r1, 2 of 2 files at r2, all commit messages.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @niklasberglund)


ios/MullvadVPNUITests/AccountTests.swift line 61 at r2 (raw file):

        LoginPage(app)
            .tapAccountNumberTextField()
            .enterText(self.hasTimeAccountNumber)

Nit: Unnecessary self

Copy link
Contributor

@buggmagnet buggmagnet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 4 of 6 files at r1, 2 of 2 files at r2, all commit messages.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @niklasberglund)


ios/MullvadVPNUITests/RelayTests.swift line 54 at r2 (raw file):

                .tapDNSSettingsCell()

            DNSSettingsPage(self.app)

If any of the steps that happens before we enable DNS ad block fails, this will put it in the wrong state for the entirety of the run.
We should be able to guarantee that it's not toggled, but set to the off state here.

I haven't tested that, but something along those lines should be good I think

    @discardableResult func toggleOffBlockAdsSwitch() -> Self {
        let blockAdsSwitch = app.cells[AccessibilityIdentifier.blockAdvertising]
            .switches[AccessibilityIdentifier.customSwitch]

        guard blockAdsSwitch.value as? String == "0" else { return self }
        blockAdsSwitch.tap()

        return self
    }

@niklasberglund niklasberglund force-pushed the fix-nightly-ios-test-issues branch 2 times, most recently from 5c98b9f to 002a255 Compare April 17, 2024 09:52
Copy link
Collaborator Author

@niklasberglund niklasberglund left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 4 of 7 files reviewed, 2 unresolved discussions (waiting on @buggmagnet and @rablador)


ios/MullvadVPNUITests/RelayTests.swift line 54 at r2 (raw file):

Previously, buggmagnet wrote…

If any of the steps that happens before we enable DNS ad block fails, this will put it in the wrong state for the entirety of the run.
We should be able to guarantee that it's not toggled, but set to the off state here.

I haven't tested that, but something along those lines should be good I think

    @discardableResult func toggleOffBlockAdsSwitch() -> Self {
        let blockAdsSwitch = app.cells[AccessibilityIdentifier.blockAdvertising]
            .switches[AccessibilityIdentifier.customSwitch]

        guard blockAdsSwitch.value as? String == "0" else { return self }
        blockAdsSwitch.tap()

        return self
    }

Have implemented a check and tapping only when the toggle is on in teardown

Copy link
Contributor

@buggmagnet buggmagnet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 3 of 3 files at r3, all commit messages.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @niklasberglund)

@niklasberglund niklasberglund force-pushed the fix-nightly-ios-test-issues branch from 002a255 to 3f22b8d Compare April 17, 2024 14:01
Copy link
Contributor

@buggmagnet buggmagnet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 1 files at r4, all commit messages.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @niklasberglund)

@buggmagnet buggmagnet merged commit f1d97bc into main Apr 17, 2024
6 of 7 checks passed
@buggmagnet buggmagnet deleted the fix-nightly-ios-test-issues branch April 17, 2024 14:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
iOS Issues related to iOS
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants